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ABSTRACT 

The  National  Agricultural  Statistics  Service  (NASS)  collects  quarterly 
data  to  provide  estimates  of  on-farm  grain  stocks  and  storage  capacity. 
A  research  project,  conducted  in  14  states  (Alabama,  Georgia,  Indiana, 
Iowa,  Louisiana,  Minnesota,  Mississippi,  Nebraska,  Ohio,  Pennsylvania, 
Tennessee,  Texas,  Virginia  and  Washington)  in  March  1988,  investigated 
the  effects  on  the  quality  of  reported  corn,  soybean,  and  all  wheat  on- 
farm  grain  stocks  and  on-farm  storage  capacity  data  of  using  previously 
collected  data  in  computer  assisted  telephone  interviews.  The  results 
showed  that  the  use  of  historical  data  from  the  December  1987 
Agricultural  Survey  as  an  editing  tool  resulted  in  some  upward 
revisions  of  initial  March  1988  and  December  1987  responses.  These 
upward  revisions  reduced  the  level  differences  between  the  survey  and 
Agricultural  Statistics  Board  balance  sheet  indications,  thus  reducing 
a  perceived  bias  in  the  survey  indications.  The  results  of  this 
research  indicate  that  the  use  of  historical  data  as  a  CATI  editing 
tool  may  improve  the  quality  of  reported  grain  stocks  data. 
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SUMMARY 


This  report  describes  an  analysis  of  the  effects  of  using  previous 
data  during  Computer  Assisted  Telephone  Interviewing  (CATI)  in  the 
collection  of  on-farm  grain  stocks  and  storage  capacity  data.  The 
objective  for  using  the  historical  data  in  collecting  current 
survey  data  is  to  improve  the  accuracy  of  on-farm  grain  stocks  and 
storage,  capacity  estimation,  while  reducing  respondent  burden  by 
minimizing  callbacks  to  clarify  questionable  reports. 

This  study  analyzed  the  effectiveness  of  using  historical  data  in 
real-time  editing  of  Agricultural  Survey  Program  (ASP)  grain 
storage  data  in  14  states  (Alabama,  Georgia,  Indiana,  Iowa, 
Louisiana,  Minnesota,  Mississippi,  Nebraska,  Ohio,  Pennsylvania, 
Tennessee,  Texas,  Virginia  and  Washington) .  Initial  March  1988  ASP 
responses  to  the  on-farm  grain  stocks  and  storage  capacity 
questions  were  subject  to  an  on-line  CATI  edit  which  compared  them 
to  December  1987  ASP  responses  to  the  same  inventory  questions. 
When  the  difference  in  the  responses  exceeded  edit  limits,  the 
respondent  was  probed  for  additional  information  to  explain  or 
reconcile  the  difference.  Both  the  current  (March  1988)  and 
previous  (December  1987)  responses  were  then  subject  to  revision, 
and  reasons  for  the  differences  which  triggered  the  edit  flags 
were  solicited.  The  aggregate  levels  of  the  revised  responses  due 
to  the  editing  for  all  stock  items  tested,  except  for  March  1988 
on-farm  soybean  and  all  wheat  stocks  for  which  few  differences  were 
flagged,  were  significantly  higher  than  the  levels  of  the  original 
responses . 

Shortly  after  collecting  the  March  1988  ASP  data  using  CATI,  a 
small  subset  of  the  respondents  in  three  states  were  contacted 
again.  Personal  face-to-face  reinterviews  were  used  to  ask  the 
same  questions  about  their  on-farm  grain  stocks  and  storage 
capacity,  with  the  same  March  1  reference  date.  The  reinterview 
responses  were  reconciled  with  the  final  CATI  responses  to 
determine  "true"  values.  For  the  three  states  (Iowa,  Nebraska  and 
Pennsylvania)  with  these  reinterview  data,  the  analysis  compares 
the  results  of  using  historical  data  in  the  editing  process  to 
these  "true"  values.  No  significant  differences  were  detected. 
This  fact  indicates  the  use  of  historical  on-farm  grain  stocks  and 
capacity  data  in  an  on-line  CATI  edit  may  not  bias  the  estimates  of 
these  items.  In  fact,  the  use  of  historical  data  in  this 
manner  resulted  in  responses  which  tended  to  be  higher  than  the 
original  responses  and  closer  to  the  reconciled  "truth". 
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INTRODUCTION 


This  report  describes  an  analysis  of  the  effects  of  using 
historical  data  in  Computer  Assisted  Telephone  Interviewing  (CATI) 
for  the  December  1987  and  March  1988  Agricultural  Survey  Program 
(ASP)  grain  stocks  data.  States  included  in  the  analysis  were 
Alabama,  Georgia,  Indiana,  Iowa,  Louisiana,  Nebraska,  Minnesota, 
Mississippi,  Ohio,  Pennsylvania,  Tennessee,  Texas,  Virginia  and 
Washington.  Variables  analyzed  were  on-farm  corn,  soybean  and  all 
wheat  stocks  and  on-farm  grain  storage  capacity.  This  was  a  two 
part  study.  The  first  part  of  the  analysis  involving  all  14  states 
investigated  the  direct  effect  on  survey  indications  of  using 
historical  data  as  an  editing  tool.  The  second  part  compared  the 
results  of  the  use  of  historical  data  in  editing  versus  "true" 
values  collected  from  a  March  1988  Reinterview  Survey  in  three  of 
the  states. 

A  split  sample  comparison  of  the  effects  of  using  historical  data 
in  editing  was  performed.  One  sample  group  (operational)  was 
interviewed  without  any  use  of  historical  data,  while  a  subgroup  of 
the  other  sample  group  (test)  was  exposed  to  the  use  of  historical 
data  as  an  editing  tool. 

As  March  1988  ASP  grain  stock  data  were  collected  in  the  test 
group,  they  were  compared  in  real-time  to  previously  reported 
December  1987  data  as  part  of  the  CATI  process.  When  a  March 
response  differed  from  the  corresponding  December  response  by  more 
than  a  predetermined  percentage,  the  respondent  was  asked  to 
explain  the  change.  This  resulted  in  revisions  of  March  and/or 
December  responses.  The  first  part  of  the  analysis  in  this  report 
is  designed  to  determine  if  the  revised  responses  are  significantly 
different  from  the  original  responses.  That  is,  if  the  differences 
[March  1988  original  -  March  1988  revised]  and  [December  1987 
original  -  December  1987  revised]  are  significantly  different  from 
zero.  Also  of  interest  was  the  direction  of  any  differences  due  to 
the  editing  with  historical  data. 

The  March  1988  CATI  sample  included  approximately  60%  of  list  units 
new  to  the  sample  in  March  and  approximately  40%  that  had  been  in 
at  least  one  quarter.  Therefore,  the  use  of  historical  data  as  an 
editing  tool  could  be  applied  to  at  most  40%  of  the  CATI  sample. 
Respondents  were  asked  to  verify  absolute  differences  between  the 
March  and  December  responses  of  at  least  10%  for  stocks  greater 
than  or  equal  to  10,000  bushels  and  at  least  50%  for  stocks  under 
10,000  bushels.  They  were  also  asked  to  verify  absolute 
differences  between  the  March  and  December  responses  of  at  least  5% 
for  storage  capacity  greater  than  or  equal  to  10,000  bushels  and  at 
least  25%  for  storage  capacity  under  10,000  bushels. 
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A  subgroup  of  March  1988  respondents  in  three  states  (IA,NE,PA) 
were  reinterviewed  shortly  after  the  survey.  Differences  between 
the  final  survey  responses  and  the  reinterview  responses  were 
reconciled  to  obtain  the  best  possible  proxies  to  truth.  The  latter 
portion  of  the  analysis  in  this  report  compares  the  results  of  the 
historical  data  editing  with  the  "true"  values  obtained  from  the 
reinterview  and  reconciliation  process  in  these  three  states. 


BACKGROUND 


Investigating  objective  and  statistically  defensible  uses  of 
historical  data  in  survey  data  collection  is  of  considerable 
interest  to  NASS.  With  the  sample  designs  used  for  NASS  surveys, 
many  of  the  same  sample  units  are  contacted  multiple  times  during 
the  survey  year.  If  previously  reported  data  can  be  used  in  a 
subsequent  survey  in  a  non-biasing  manner,  these  data  can  perhaps 
be  exploited  to  improve  survey  results.  This  report  documents  the 
results  of  research  on  the  use  of  historic  grain  stocks  data  for 
editing  in  a  CATI  interview.  The  research  was  initiated  to 
investigate  the  use  of  historical  data  in  improving  the  quality  of 
the  survey  data  collected,  while  reducing  respondent  burden. 

The  potential  for  improvement  in  survey  data  quality  lies  in  the 
fact  that  indicated  survey  increases  or  decreases  in  stocks  and/or 
capacity  between  quarters  are  often  not  indicative  of  real  change. 
Differences  can  be  due  to  a  definitional  problem,  such  as  a 
respondent's  failure  to  include  all  stocks  on  the  total  acres 
operated  and  exclude  stocks  stored  elsewhere,  or  simply  to  a  change 
in  respondent  between  quarters.  These  situations,  when  they  can  be 
identified,  often  prompt  a  return  call  to  the  respondent  to  clarify 
unexpected  responses.  An  on-line  CATI  edit  allows  responses 
outside  an  expected  range  to  be  reconciled  during  the  initial 
interview.  Respondent  burden  is  reduced  by  contacting  the 
respondent  just  once  during  the  survey,  and  the  efforts  of  the 
office  staff  can  then  be  directed  to  areas  other  than  callbacks. 

The  1988  March  Agricultural  Survey  (MAS)  was  selected  for  this 
research  since  declines  in  on-farm  grain  stocks  are  normal  between 
December  and  March.  The  expected  changes  in  the  stocks  stored 
during  this  period  would  cause  the  comparison  between  the  current 
and  historical  responses  to  frequently  fall  outside  of  the  edit 
range,  triggering  the  additional  question (s)  to  be  asked. 
Historical  data  were  provided  after  an  edit  check  in  order  to  avoid 
biases  from  enumerators  influencing  respondents  with  prior 
responses . 
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The  use  of  historical  data  in  an  interview  situation  has  not  been 
extensively  explored.  0 'Muircheartaigh  (1986)  provided  initial 
interview  information  on  paper  to  enumerators  conducting 
reinterviews.  He  found  that  instructions  for  enumerators  not  to 
utilize  the  historical  data  prior  to  the  completion  of  the 
reinterview  were  not  followed.  Bailar  (1968)  also  studied  the  use 
of  historical  data  and  the  reinterview  process.  She  states  that  the 
best  procedures  for  evaluating  the  quality  of  a  survey  include  a 
reinterview  relatively  close  in  time  to  the  original  interview, 
with  reinterviewers  not  having  access  to  the  original  responses 
until  the  reinterview  is  completed. 

Within  NASS,  the  results  of  the  direct  use  of  historical  data  in  an 
interview  (Pafford  1986,  1988)  and  the  use  of  historical  data  in 
the  margins  of  the  CATI  screen  (Pafford  1986)  showed  that  these 
methods  have  the  potential  to  bias  the  respondent's  answers. 
Steiner  (1980)  looked  at  differences  between  samples  of  telephone 
respondents  and  respondents  interviewed  face-to-face  when 
historical  pig  farrowing  information  was  provided  by  the 
enumerator.  Results  indicated  some  effect  on  telephone 
respondents,  but  no  effect  on  those  interviewed  face-to-face.  House 
(1984)  stated  that  improvements  in  data  quality  can  be  gained  by 
CATI  on-line  edit  checks  for  data  consistency  within  an  interview, 
or  when  responses  are  compared  to  previously  reported  control  data. 


INSTRUMENT  DESIGN 


One  CATI  instrument  was  developed  with  separate  sections  to  handle 
the  operational  and  treatment  designs  for  the  fourteen  CATI  states 
in  the  research  project.  An  example  of  each  of  the  CATI 
operational  and  treatment  grain  stocks  sections  is  presented  in 
Appendix  A. 

In  the  operational  design,  respondents  were  asked  about  storage  on 
their  operation  of  the  grains  of  interest  to  the  state  (in  Indiana, 
for  example,  corn,  soybeans,  and  wheat) ,  and  about  the  total 
storage  capacity  of  the  operation. 

While  the  treatment  design  asked  these  same  questions,  it  differed 
from  the  operational  design  in  the  addition  of  several  questions 
which  appeared  to  the  enumerator  when  on-line  edit  checks  failed. 
The  edit  checks  compared  the  current  (March  1,  1988)  reported  grain 
stocks  data  to  the  historical  (December  1,  1987)  grain  stocks  data 
for  the  operation.  Potential  errors  were  flagged  when  all  of  the 
following  conditions  were  met: 

The  respondent  was  in  the  December  survey; 

Neither  the  December  or  March  stocks  were  missing; 


1) 

2) 
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3)  There  was  a  large  increase  or  decrease  in  the  reported 
commodity  specific  stocks  or  capacity  between  December 
and  March.  Large  increases  or  decreases  were  defined  as: 

a)  differences  of  more  than  10%  of  December 
reported  stocks  for  stocks  of  10,000  or  more 
bushels ; 

b)  differences  of  more  than  50%  of  December  reported 
stocks  for  stocks  under  10,000  bushels; 

c)  differences  of  more  than  5%  of  December  reported 
capacity  for  storage  capacity  of  10,000  or  more 
bushels ; 

d)  differences  of  more  than  25%  of  December  reported 
capacity  for  storage  capacity  under  10,000  bushels. 


CATI  probing  occurred  immediately  after  the  completion  of  the  grain 
stocks  section.  This  ordering  of  questions  enabled  us  to  explore 
the  possibility  of  improving  current  grain  stocks  responses  using 
historical  data  without  biasing  initial  survey  responses  by 
"feeding"  respondents  previous  responses. 


THE  SAMPLE 


A  split  sample  design  was  utilized  to  evaluate  the  use  of  a  prior 
quarter's  grain  stocks  data  as  an  edit  check  during  a  grain  stocks 
survey.  One  group  (operational)  consisted  of  sample  units  new  to 
the  survey.  The  other  group  (test)  consisted  of  sample  units  which 
had  been  included  in  the  prior  quarter's  survey.  The  operational 
or  control  group  had  no  access  to  historical  data.  The  test  group 
sample  units  were  provided  the  historical  data  when  edit  checks 
exceeded  the  CATI  edit  limits.  A  layout  of  the  sample  design  used 
in  each  state  is  presented  below: 


CONTROL 

TE 

ST 

ORIGINAL 

REVISED 

March  1988 

March  1988 

Dec.  1987 

March  1988 

Dec.  1987 
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Responses  to  wheat,  corn  and  soybean  on-farm  grain  storage  and 
total  storage  capacity  were  obtained  in  March  1988  and  compared  to 
responses  obtained  in  December  1987.  Revised  responses  were 
obtained  for  the  test  group  units  whose  data  comparison  fell 
outside  of  specified  edit  limits.  The  revised  March  1988  and 
December  1987  responses  were  compared  to  corresponding  original 
responses  to  evaluate  the  significance  of  differences. 

The  study  utilized  only  the  list  portion  of  the  ASP  sample.  Sample 
units  were  randomly  selected  within  strata  designated  for  CATI 
enumeration . 


RESULTS 


Since  prior  research  has  shown  different  levels  of  estimates  due  to 
the  number  of  times  sample  units  have  previously  been  contacted  in 
the  survey  program,  analysis  was  performed  to  determine  if  there 
was  a  rotation  group  effect.  This  is  not  the  primary  purpose  of 
this  study;  however,  since  all  operational  group  samples  were  new 
samples  and  all  test  group  samples  were  "old"  samples,  differences 
detected  during  analysis  of  the  test  group  data  may  be  confounded 
with  a  rotation  group  effect.  Table  1  lists  approximate  95% 
confidence  intervals  of  mean  March  1988  grain  stocks  (corn, 
soybeans  and  all  wheat)  and  storage  capacity  responses  by  group 
(operational  and  test) .  Since  the  intervals  do  not  overlap,  we  can 
conclude  that  there  is  a  rotation  group  effect,  and  that  "new" 
samples  tended  to  result  in  lower  reports  than  those  that  had  been 
contacted  at  least  one  previous  quarter.  This  result  somewhat 
limits  the  scope  of  inference  of  other  analyses  using  sample  units 
in  the  test  group.  That  is,  any  differences  detected  may  be 
confounded  with  the  number  of  times  sample  units  have  been 
previously  contacted  in  the  survey  program. 


Table  1.  Historical  Data  Analysis  (March  1988  Responses) : 

A  comparison  of  operational  and  test  group  mean  responses 

95%  confidence  intervals  (units  are  bushels) . 

Storage  Item 

Record  Counts 

Operational 

Interval 

Test  Interval 

Lower 

Limit 

Upper 

Limit 

Lower 

Limit 

Upper 

Limit 

Oper . 

Test 

Corn 

7 , 623 

4 , 549 

3 , 170 

3 , 650 

3 , 850 

4 , 510 

Soybeans 

7 , 683 

4 , 577 

320 

380 

380 

490 

All  Wheat 

7 ,716 

4 , 590 

60 

100 

100 

230 

Capacity 

7 , 527 

4 ,489 

8 ,850 

9, 350 

9,600 

10, 310 
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The  purpose  of  the  next  part  of  this  analysis  was  to  determine  if 
the  edited  responses  were  significantly  different  from  the  original 
responses.  Of  the  March  1988  responses  which  were  changed  due  to 
the  edit,  the  majority  of  the  differences  between  the  original  and 
the  revised  responses  for  corn  and  capacity  were  negative.  This 
indicates  UPWARD  revisions  of  these  March  1988  responses.  Sign 
tests  on  these  differences  were  performed  to  determine  if  the 
number  of  upward  revisions  were  significantly  different  from  the 
number  of  downward  revisions.  Significant  differences  were 
detected  for  corn  stocks  and  capacity.  The  number  of  revisions  to 
the  March  1988  on-farm  soybean  and  all  wheat  storage  responses  were 
very  small  and  did  not  provide  much  power  to  detect  any 
differences.  Of  the  December  1987  responses  which  were  changed  due 
to  the  edit,  the  majority  of  the  differences  between  the  original 
and  the  revised  responses  were  also  negative,  indicating  UPWARD 
revisions.  Sign  tests  on  these  differences  showed  that  the  number 
of  upward  revisions  to  the  December  1987  responses  was 
significantly  greater  than  the  number  of  downward  revisions  for  all 
four  analysis  variables. 

In  this  analysis  the  sign  tests  provide  an  overall  indication  of 
the  direction  of  the  revisions.  However,  they  do  not  provide  a 
precise  nor  statistically  sound  measure  of  the  magnitude  of  the 
differences.  Design  based  means  and  standard  errors  of  the 
differences  were  computed  to  provide  statistical  measures  of  the 
differences  and  to  evaluate  their  significance. 

Table  2  lists  by  storage  item  the  number  of  usable  records  in  the 
test  domain  which  triggered  an  edit  check,  mean  differences  between 
original  March  1988  and  edited  March  1988  responses,  and  95% 
confidence  intervals  for  the  differences.  Since  zero  is  not 
covered  by  the  confidence  intervals  for  on-farm  corn  stocks  and 
storage  capacity,  we  can  conclude  that  the  mean  difference  between 
original  and  edited  responses  is  not  zero  for  these  analysis 
variables.  No  significant  differences  were  detected  for  on-farm 
soybean  or  all  wheat  stocks,  but  too  few  of  these  variables  were 
revised  to  have  sufficient  power  to  detect  any  differences. 
Results  indicate  the  use  of  historical  data  as  an  editing  tool 
resulted  in  an  upward  revision  of  original  March  1988  on-farm  corn 
stocks  and  grain  storage  capacity  responses. 
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Table  2.  Historical  Data  Analysis:  (March  Original  -  March  Edited) 
95%  Confidence  Interval  for  Difference  and  Sign  Test  p-values. 

Storage  Item 

Record 

Counts 

Mean 

Lower 

Limit 

Upper 

Limit 

Sign  Test 
p-value 

Corn 

1,289 

-150 

-300 

0 

0.01 

Soybeans 

528 

15 

-20 

50 

0.11 

All  Wheat 

242 

-15 

-50 

20 

0.18 

Capacity 

1,632 

-475 

-690 

-260 

<  0.01 

Table  3  lists  by  storage  item  the  number  of  usable  records  in  the 
test  domain  which  triggered  an  edit  check,  mean  differences  between 
original  and  edited  December  1987  responses,  and  95%  confidence 
intervals  for  the  differences.  Since  the  confidence  intervals  do 
not  contain  zero  we  can  conclude  that  the  mean  difference  between 
original  and  edited  December  1987  responses  is  not  equal  to  zero. 
These  December  results,  like  some  of  the  March  results,  indicate 
the  use  of  historical  data  as  an  editing  tool  resulted  in  an  upward 
revision  of  original  responses. 


Table  3.  Historical  Data  Analysis:  {December  (Original  -  Edited)} 

95%  Confidence  Interval  for  Difference  and  Sign  Test  p-values. 

Storage  Item 

Record 

Counts 

Mean 

Lower 

Limit 

Upper 

Limit 

Sign  Test 
p-value 

Corn 

1,289 

-570 

-870 

-270 

<  0.01 

Soybeans 

528 

-165 

-320 

-10 

<  0.01 

All  Wheat 

242 

-85 

-135 

-35 

<  0.01 

Capacity 

1,632 

-460 

-810 

-110 

<  0.01 
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Table  4  lists  reasons  for  the  differences  between  original  March 
1988  and  December  1987  responses  and  the  percent  of  responses  of 
each  reason  by  grain  storage  summary  item.  Most  of  the  differences 
were  due  to:  1)  stocks  being  bought  or  sold,  2)  stocks  being 
estimated  or  incorrect,  3)  stocks  being  fed  to  livestock  or  4) 
storing  another  person's  grain. 


Table  4.  Historical  Data  Analysis:  Reasons  For  Differences  Between 
Original  March  Responses  and  Original  December  Responses. 

Reason 

Percentages 

Corn 

Soybeans 

Wheat 

Capacity 

Bought/Sold 

33 

59 

38 

14 

Dec/Mar  stocks  were 
estimated  or  incorrect 

18 

15 

16 

52 

Fed  livestock  or  storing 
other's  grain 

27 

3 

10 

2 

Failed  to  include 
reserved/sealed  grain 

2 

1 

2 

0 

Failed  to  include  all  stocks 
(facilities)  on  land 

3 

1 

2 

10 

Crop  stored  off  farm 

2 

3 

2 

0 

Unable  to  resolve  reason 

7 

10 

16 

9 

Other 

8 

8 

14 

13 

The  purpose  of  the  remainder  of  the  analysis  was  to  compare  the 
results  of  the  use  of  historical  data  in  editing  to  "true"  values 
collected  from  a  March  1988  Reinterview  Survey  conducted  in  three 
of  the  test  states  (IA,  NE  and  PA) .  The  reinterview  sample  which 
was  drawn  randomly  from  all  March  1988  list  samples  completed  on 
CAT I ,  spanned  the  operational  and  test  domains  of  this  research. 
For  our  purposes  here  the  respondents  subject  to  the  use  of 
historical  data  were  restricted  to  the  test  domain,  so  that  only 
reinterview  samples  falling  in  this  domain  could  be  used  in  the 
comparisons.  Therefore,  results  presented  below  should  not  be 
compared  to  or  confused  with  results  presented  in  the  report  by 
McClung  and  others  which  benefited  from  the  full  reinterview  sample 
sizes . 
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Table  5  summarizes  comparisons  of  March  1988  final  responses  to  the 
proxies  to  the  truth.  Listed  by  storage  item  are  the  number  of 
records  which  both  triggered  an  edit  verification  and  had  a 
reconciled  "true"  value,  the  mean  differences  between  the  March 
1988  final  responses  and  the  "true"  value,  and  95%  confidence 
intervals  for  the  differences.  Since  zero  fell  within  the 
confidence  intervals  of  the  mean  differences  for  on-farm  soybean 
and  all  wheat  storage  and  on-farm  grain  storage  capacity,  we  can 
conclude  that  the  mean  difference  is  not  significantly  different 
from  zero  for  these  analysis  variables.  However,  the  mean 
difference  is;  significantly  different  from  zero  for  on-farm  corn 
stocks.  Sign  tests  on  the  differences  were  performed  to  determine 
if  the  number  of  positive  differences  was  greater  than  the  number 
of  negative  differences.  A  significant  number  of  the  proxies  were 
at  higher  levels  than  the  corresponding  final  responses  for  on-farm 
corn  and  soybean  stocks.  However,  the  historical  data  editing  did 
tend  to  bring  final  responses  closer  to  the  "true"  values. 


Table  5.  Historical  Data  Analysis:  Mean  Differences  Between  the 

Proxy  to  the  Truth  and 

the  Final  Response  of  March  1988  Respondents 

Subject  to 

the  Historical  Data  Edit,  95%  Confidence  Intervals  for 

the  Differences  and  One  Tail  Sign 

Test  p-values. 

Storage 

Record 

Lower 

Upper 

Sign  Test 

Item 

Counts 

Mean 

Limit 

Limit 

p-value 

Corn 

172 

1020 

0 

2 , 040 

0.02 

Soybeans 

173 

180 

-70 

430 

0.01 

All  Wheat 

173 

20 

-25 

75 

0.50 

Capacity 

169 

105 

-1,455 

1,665 

0.20 

The  analyses  indicate  the  use  of  historical  data  as  an  editing  tool 
results  in  responses  which  tend  to  be  higher  than  the  original 
responses  but  less  than  the  "truth". 
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DISCUSSION  &  RECOMMENDATIONS 


The  use  of  prior  stocks  data  as  an  editing  tool  has  been  frequently 
proposed  within  NASS.  For  instance,  as  this  project  was  being 
developed  the  Nebraska  State  Statistical  Office  (SSO)  suggested 
that  previous  quarter's  stocks  data  be  made  available  to  their 
enumerators.  This  suggestion  stemmed  from  their  experience  on  the 
1987  March  Agricultural  Survey,  where  overnight  computer  edits 
identified  numerous  cases  of  stocks  increases  from  the  1986 
December  Agricultural  Survey.  When  called  back  to  clarify  the 
situation,  respondents  invariably  corrected  their  responses.  The 
Nebraska  staff  felt  that  having  the  respondent's  previous  quarter 
data  available  in  the  current  survey  would  eliminate  callbacks  to 
many  operations. 

Reaction  to  the  project  was  favorable  in  the  Indiana  SSO.  After 
reconciling  potential  errors  in  the  grain  stocks  section, 
enumerators  often  joked  with  the  respondents  that  they  would  not 
be  checking  up  on  any  more  of  the  respondent's  replies.  This  grew 
from  one  enumerator's  experience  that  it  seemed  to  keep  the 
respondent  at  ease  throughout  the  interview.  Also,  statisticians  in 
the  SSO  felt  that  the  good  comments  the  enumerators  attached  to 
unusual  but  accurate  situations  in  the  grains  stocks  section  saved 
two  full  days  of  callbacks.  They  appreciated  the  immediate 
correction  of  detected  errors,  improving  the  quality  of  the  data 
collected . 

Comments  from  the  Texas  SSO  were  somewhat  less  favorable.  They 
indicated  that,  while  the  use  of  historical  data  seemed  to  help  in 
some  cases,  there  was  significant  concern  among  respondents  that  we 
had  access  to  their  historical  data.  Some  respondents  indicated 
that  they  had  been  persuaded  to  respond  in  earlier  surveys  by  being 
told  that  all  questionnaires  were  shredded  soon  after  the  survey. 

Based  on  the  results  of  these  analyses  it  appears  that  the 
operational  use  of  historical  data  in  the  context  of  CATI  editing 
as  described  in  this  report  is  probably  appropriate  and  may  well 
improve  the  quality  of  reported  grain  storage  data.  The  analyses 
indicated  that  this  use  of  historical  data  (quite  possibly  our  most 
statistically  defensible  use  of  these  data)  may  help  to  reduce  an 
indicated  downward  bias  in  our  grain  storage  reporting. 

One  factor  to  consider  in  implementing  the  approach  operationally 
is  to  determine  an  appropriate  editing  or  tolerance  level  for 
further  probing  for  each  item.  For  a  particular  survey  this  level 
can  best  be  determined  by  reviewing  past  survey  data  for  the  levels 
of  change  that  do  routinely  exist  from  quarter  to  quarter. 

Some  other  general  considerations  in  the  Agency's  use  of  historical 
data  are  the  following: 
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*  Avoid  any  use  of  historical  data  in  which  the  respondent  is 
"fed"  a  previous  response  prior  to  providing  a  current  survey 
response.  Previous  survey  research  has  indicated  that  this 
approach  to  using  historical  data  biases  reported  data. 

*  Limit  the  use  of  historical  data  edit  checks  in  a  CATI 
interview.  Concentrating  on  grain  stocks,  or  hog  farrowings,  or 
cropland  acreages  would  be  preferable  to  an  entire  interview  with 
current  levels  checked  against  historical  data.  The  "entire" 
approach  would  be  tedious  and  lead  to  comments  of  "If  you  already 
know  all  of  this,  why  are  you  asking  me?",  or  "Just  put  the  same  as 
last  time." 

*  Ask  permission  of  the  respondent  in  the  base  period  for  the 
enumerator  to  be  allowed  to  repeat  back  to  the  respondent  (or 
others  as  they  may  designate)  specific  pieces  of  information  in 
subseguent  interviews.  Statisticians  would  know  in  advance  when  an 
historical  data  edit  check  would  be  used,  and  we  should  let  the 
respondent  in  on  this,  enlisting  their  cooperation  in  producing  the 
highest  quality  estimates  possible. 

By  limiting  our  requests  to  certain  bits  of  information  in  a  well 
defined  time  span,  we  should  be  able  to  minimize  respondent 
hesitation  to  this  concept.  And  by  relating  back  base  period  data 
only  to  the  same  respondent  (or  others  as  they  may  designate) ,  our 
pledge  of  confidentiality  can  be  maintained. 

*  Be  sure  to  add  an  enumerator  comment  to  the  interview  after  the 
section  in  which  an  historical  data  check  is  made,  indicating  that 
no  more  checks  of  this  type  will  occur  in  the  interview.  This 
would  still  allow  for  other  edit  checks  comparing  responses  given 
within  the  current  interview. 


Current  plans  are  for  the  Nonsampling  Errors  Section  to  investigate 
the  effect  of  historical  data  editing  on  acreage  indications.  This 
investigation  will  probably  be  tied  to  the  1992  September 
Agricultural  Survey  with  June  1992  survey  data  used  for  historical 
data  edits  in  several  states. 
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APPENDIX  A.  Computer  Assisted  Telephone  Interviewing  (CATI)  Screens 

Part  I.  Operational  Grain  Stocks  Section  for  Indiana 

1988  March  Agricultural  Survey 


How  much  WHOLE  GRAIN  CORN  from  1987  and  earlier  crop  years  was 
in  storage  on  March  1st  on  this  operation? 

<0>  NONE  [advances  to  next  grain  type] 

<l-9999999>  Bushels,  Tons,  Lbs,  Cwt 

[advances  to  next  grain  type] 

<*>  CROP  IN  STORAGE  BUT  AMOUNT  UNKNOWN 

[continues] 

Could  you  give  me  your  best  estimate?? 

<0>  NONE  [continues] 

<l-9999999>  Bushels,  Tons,  Lbs,  Cwt 

[continues] 

<*>  NO  IDEA  [continues] 
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How  much  SOYBEANS  from  1987  and  earlier  crop  years  was  in 
storage  on  March  1st?? 


<0> 

<l-9999999> 

<*> 


NONE  [advances  to  next  grain  type] 
Bushels,  Tons,  Lbs,  Cwt 
[advances  to  next  grain  type] 

NOT  KNOWN  [continues] 


Could  you  give  me  your  best 

<0> 

<l-9999999> 

<*> 


estimate?? 

NONE  [continues] 
Bushels,  Tons,  Lbs,  Cwt 
[continues] 

NO  IDEA  [continues] 


How  much  WINTER  WHEAT  was  in  storage  on  March  1st  on  this 
operation??  Include  1987  and  earlier  crop  years. 

<0>  NONE  [advances  to  TOTAL  STORAGE  CAPACITY] 

<l-9999999>  Bushels,  Tons,  Lbs,  Cwt 

[advances  to  TOTAL  STORAGE  CAPACITY] 

<*>  NOT  KNOWN  [continues] 


Could  you  give  me  your  best  estimate?? 


<0> 

<l-9999999> 

<*> 


NONE  [continues] 
Bushels,  Tons,  Lbs,  Cwt 
[ continues ] 

NO  IDEA  [continues] 
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What  was  the  TOTAL  STORAGE  CAPACITY  of  all  bins,  cribs,  sheds 
and  other  structures  normally  used  to  store  whole  grains  and 
oilseeds  on  those  f current  interview  reportedl  acres?? 


<0-9999999>  Bushels,  Tons,  Lbs,  Cwt 


<k> 

[advances  to  next  section] 

NO  ANSWER  [advances  to  next  section] 

[Edit  Check: 
capacity,  the 

when  reported  stocks  are  greater  than  reported 
following  message  is  displayed.] 

SUM  OF  GRAIN  STORED  ITEMS  IS  GREATER  THAN  GRAIN  STORAGE 
CAPACITY. 

GRAIN  STORAGE  CAPACITY  \ current  bushel  equiv. 1  f current  cap.  1 
SUM  OF  GRAIN  STORED  \ current  bushel  equiv. 1  f current  total! 


CORN. . . . 
SOYBEANS. 

JUMPBACK  TO  CORRECT 

ALL  WHEAT . [current  interview  reportedl  Bushels 


CAPACITY. 

PUSH  <1> 

if  ANSWER  is  CORRECT 

[advances  to  next  section] 

15 


APPENDIX  A.  Computer  Assisted  Telephone  Interviewing  (CATI)  Screens 
Part  II.  Treatment  Corn  Stocks  Section  for  Indiana 


Next,  we  would  like  to  talk  about  grains  and  oilseeds  being 
stored  on  the  [current  interview  reported 1  acres  operated  by 
[operation  namel .  whether  for  feed,  seed  or  sale.  They  can 
belong  to  you  or  someone  else  or  be  under  government  loan, 
farmer  owned  reserve  or  CCC  program. 

On  March  1st,  did  you  have  any  of  the  following  grains  or 
oilseeds  in  storage? 

<CORN 

<SOYBEANS 

<WHEAT 

<1>  YES  [continues] 

< 3 >  NO  [advances  to  TOTAL  STORAGE  CAPACITY  question] 

<k>  DON'T  KNOW  [advances  to  next  section] 

<r>  REFUSED  GRAIN  STOCKS  SECTION  [advances  to  next  section] 


How  much  WHOLE  GRAIN  CORN  from  1987  and  earlier  crop  years  was 
in  storage  on  March  1st  on  this  operation? 

<0>  NONE  [compares  current  response  to  stored 

December  1  response] 

<l-9999999>  Bushels,  Tons,  Lbs,  Cwt  [compares  current 

response  to  December  1  response] 

<*>  CROP  IN  STORAGE  BUT  AMOUNT  UNKNOWN 

[ continues ] 

Could  you  give  me  your  best  estimate?? 


<0>  NONE  [compares  current  response 

to  December  1  response] 

<l-9999999>  Bushels,  Tons,  Lbs,  Cwt  [compares 

current  response  to  December  1  response] 
<*>  NO  IDEA  [advances  to  next  grain  type] 


[The  program  continues  on  through  the  other  grains  of  interest  to 
the  state  (in  Indiana,  for  example,  soybeans  and  wheat)  and  through 
the  Total  Storage  Capacity  question.  Then,  for  each  grain  stored 
which  had  a  <0>  or  <l-9999999>,  the  current : historic  edit  check  is 
invoked.  For  each  grain  in  which  the  proper  conditions  are  met, 
the  following  statement  is  displayed  (continuing  the  example  with 
CORN) : ] 
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Our  records  show  that  your  operation  reported  f  December 
interview  reported!  Bushels  of  CORN  being  stored  as  of  December 
1.  We  now  have  f current  interview  reported 1  Bushels! 

Can  you  describe  the  reason  for  this  increase/decrease  [case 
specific] ? 

<1>  Bought/sold  [case  specific]  CORN  [advances  to  next 

grain  type] 

<2>  December/March  stocks  were  estimated/incorrect 

[ continues ] 

<3>  Storing  someone  else's  corn/fed  to  livestock  [case 
specific]  [continues] 

<4>  December/Current  [case  specific]  failed  to  include 
RESERVE/SEALED  Corn  [continues] 

<5>  Failed  to  include  ALL  stocks  on  land  operated 
[ continues ] 

<6>  CORN  stored  in  OFF  FARM  FACILITY;  i.e.  local  elevator, 
etc.  [continues] 

<7>  Unable  to  resolve  reason  for  difference  [go  to  XXa] 

<8>  OTHER,  specify  [continues] 


Could  you  provide  the  CORRECT  data  for  CORN  stored  on  this 
operation? 


************************************************************** 
CURRENT  STOCKS  REPORTED  =  [ current  interview  reported!  Bushels 
DECEMBER  REPORTED  STOCKS  =  [Dec,  interview  reported]  Bushels 
************************************************************** 

<1>  to  correct  Current  stocks  [continues] 

<2 >  to  correct  December  stocks  [continues] 

<3>  to  correct  BOTH  [continues] 

<9>  ALL  ENTRIES  CORRECT  [advances  to  next  grain  type] 
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[IF  <1>  OR  <3>:] 


What  is  the  correct  amount  of  CORN  in  storage  March  1st? 

★★★★Reported  March  1  =  [current  interview  reported!  Bu.**** 


<0-9999999> 

<k> 


BUSHELS,  TONS,  LBS,  CWT 
UNKNOWN 


[IF  <2 >  OR  <3 >  S ] 

What  was  the  correct  amount  of  CORN  in  storage  December  1,  1987? 

★★★★Reported  December  1  =  [Dec,  interview  reported!  Bu.**** 

<0-9999999>  BUSHELS,  TONS,  LBS,  CWT 

<k>  UNKNOWN 


XXa 


[As  dictated  by  the  on-line  edit,  the  screen  advances  to 
inquire  further  about  the  next  grain  type  or  Total  Storage 
Capacity,  or  on  to  the  next  section.] 
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APPENDIX  A.  Computer  Assisted  Telephone  Interviewing  (CATI)  Screens 
Part  III.  Treatment  Storage  Capacity  Section  for  Indiana 


What  was  the  TOTAL  STORAGE  CAPACITY  of  all  bins,  cribs,  sheds 
and  other  structures  normally  used  to  store  whole  grains  and 
oilseeds  on  those  [current  interview  reported!  acres?? 

<0-9999999>  Bushels,  Tons,  Lbs,  Cwt  [advances  to  next 

section] 

<k>  NO  ANSWER  [advances  to  next  section] 


[If  the  <0-9999999>  response  does  not  trigger  an  edit  check, 
the  screen  advances  to  the  next  section.  If  any  conditions 
which  trigger  an  edit  check  are  met,  the  following  statement 
is  displayed:] 


Our  records  show  that  your  operation  reported  f  December 
interview  reported!  Bushels  of  STORAGE  CAPACITY  as  of  December 
1.  We  now  have  f current  interview  reported  1  Bushels! 


Can  you  describe  the  reason  for  this  increase/decrease  [case 
specific] ? 


<1>  Added/deleted  [case  specific]  FACILITIES  [advances  to 

next  section] 

<2 >  December/March  CAPACITY  was  estimated/incorrect 

[continues] 

<3 >  Rented  additional  facility/facility  destroyed  [case 
specific]  [advances  to  next  section] 

<4 >  Failed  to  include  ALL  FACILITIES  on  land  operated 

[ continues ] 

<5>  Unable  to  resolve  reason  for  difference  [go  to  XXb] 

<6>  Other  (specify)  [continues] 


Could  you  provide  the  CORRECT  data  for  STORAGE  CAPACITY  on  this 
operation? 


★  ★A****************************************'*'*'****'*'*'***'*'********'* 

CURRENT  CAPACITY  REPORTED=  [ current  interview  reported]  Bushels 
DECEMBER  CAPACITY  REPORTED  =  \ Dec .  interview  reported!  Bushels 
*****************************■*******************'*****•*********** 


<1> 

to 

correct 

Current 

<2  > 

to 

correct 

December 

<3> 

to 

correct 

BOTH 

<9> 

ALL  ENTRIES 

CORRECT 

Stocks  [continues] 

Stocks  [continues] 

[ continues ] 

[advances  to  next  section] 
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[IF  <1>  OR  <3> : ] 


What  is  the  correct  amount  of  CAPACITY  on  March  1st? 

★★★★Reported  March  1  =  \ current  interview  reported 1  Bushels**** 

<0-9999999>  BUSHELS,  TONS,  LBS,  CWT 

<k>  UNKNOWN 


[IF  <2 >  OR  <3> : ] 

. . in  m  in*— wiw— 

What  was  the  correct  amount  of  STORAGE  CAPACITY  on  December  1st, 
1987? 

****Reported  Dec.  1  =  r December  interview  reportedl  Bushels**** 

<0-9999999>  BUSHELS,  TONS,  LBS,  CWT 

<k>  UNKNOWN 


XXb 


[The  screen  advances  to  next  section,  unless  reported  stocks  are 
greater  than  reported  capacity.  Then  the  following  question  is 
displayed: ] 
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The  FINAL  reported  grains  in  storage  are  greater  than  the 
CAPACITY! ! 

Let's  review  what  I  have  recorded: 

STORAGE  CAPACITY . [current  interview  reported  1  Bushels 

TOTAL  STOCKS . \ current  interview  reported]  Bushel  Equiv. 

CORN  STOCKS . f current  interview  reported!  Bushels 

SOYBEAN  STOCKS . [current  interview  reported!  Bushels 

WHEAT  STOCKS . \  current  interview  reported  1  Bushels 


ENTER  <1>  THEN  please  enter  the  correct  STOCKS  and  CAPACITY  in 
notes  ===> 


[The  screen  advances  to  the  next  section.] 
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